Projection adjustment program and projection adjustment method

ABSTRACT

A non-transitory computer-readable storage medium stores a projection adjustment program which, when executed by a processor, causes a computer to execute a process relating to adjustment of projection operations of projection devices configured to perform position measurement and projection on a target object in a projection system including the projection devices. The process includes: causing a first projection device to project invisible measurement light onto the target object; causing a second projection device to receive reflection light reflected from the target object; judging a connection relationship of a projection range of the first projection device on the basis of the received reflection light; executing a process of the judging of the connection relationship on all processing target projection devices; and generating projection position information indicating a connection relationship between projection ranges of the respective projection devices and displaying the projection position information on a display unit.

TECHNICAL FIELD

The present disclosure relates to a projection adjustment program and aprojection adjustment method used in a projection system for projectinga video onto a target object.

BACKGROUND ART

A technology for projecting a video onto a target object such as ascreen or a construction, a technology called “projection mapping,” isknown. Among projection mapping systems are systems having an imagecapture function. For example, Patent Literature 1 discloses a systemcapable of acquiring a 3D shape of a subject and capturing an image ofthe subject with visible light at the same time.

Various projection systems using a plurality of projection devices havebeen proposed for purposes of large-screen display etc. Among projectionsystems of this kind are a multi-projection system which performslarge-screen display by arranging a plurality of projection devices inthe horizontal direction and the vertical direction and displayingprojected pictures of the respective projection devices side by side anda stack projection system which increases the brightness of a projectedpicture by displaying projected pictures of respective devices in asuperimposed manner. For example, Patent Literature 2 discloses a systemwhich makes it possible to manipulate each projector easily or tomanipulate all projectors together easily by performing an infraredcommunication between the plurality of projectors.

CITATION LIST Patent Literature

Patent Literature 1: JP-A-2005-258622

Patent Literature 2: WO 2011/001507 A1

SUMMARY OF INVENTION Technical Problem

An object of the present disclosure is to provide a projectionadjustment program and a projection adjustment method which allow, inarranging a plurality of projection devices, a user to easily recognizea positional relationship between a plurality of projection ranges.

The disclosure provides a projection adjustment program which causes acomputer to execute a process relating to adjustment of projectionoperations of a plurality of projection devices configured to performposition measurement and projection on a target object in a projectionsystem including the plurality of projection devices, the processincluding: causing a first projection device of the projection system toproject invisible measurement light onto the target object; causing asecond projection device of the projection system to receive reflectionlight of the measurement light, the reflection light being reflectedfrom the target object; judging a connection relationship of aprojection range of the first projection device on the basis of thereceived reflection light of the measurement light; executing a processof the judging of the connection relationship on all processing targetprojection devices; and generating projection position informationindicating a connection relationship between projection ranges of therespective projection devices of the projection system and displayingthe projection position information on a display unit.

The disclosure also provides a projection adjustment method of aprojection adjustment device configured to execute a process relating toadjustment of projection operations of the plurality of projectiondevices configured to perform position measurement and projection on atarget object in a projection system including the plurality ofprojection devices, the projection adjustment method including: causinga first projection device of the projection system to project invisiblemeasurement light onto the target object; causing a second projectiondevice of the projection system to receive reflection light, reflectedfrom the target object, of the measurement light; judging a connectionrelationship of a projection range of the first projection device on thebasis of the received reflection light of the measurement light;executing the connection relationship judging step on all processingtarget projection devices; and generating projection positioninformation indicating a connection relationship between the projectionranges of the respective projection devices of the projection system anddisplaying the generated projection position information on a displayunit.

Advantageous Effects of Invention

The disclosure makes it possible to allow, in arranging a plurality ofprojection devices, a user to easily recognize a positional relationshipbetween a plurality of projection ranges.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram for outlining of the configuration and function of ameasuring projection device according to an embodiment.

FIG. 2 is a diagram showing a first example use of a projection systemaccording to the embodiment.

FIG. 3 is a diagram showing a second example use of a projection systemaccording to the embodiment.

FIG. 4 is a diagram showing a third example use of a projection systemaccording to the embodiment.

FIG. 5 is a diagram showing a first example configuration of aprojection system according to the embodiment.

FIG. 6 is a diagram showing a second example configuration of aprojection system according to the embodiment.

FIG. 7 is a diagram showing a rough configuration of a measuringprojection device according to the embodiment.

FIG. 8 shows example invisible light measurement patterns in theembodiment.

FIG. 9 is a diagram showing the configuration of a measuring projectiondevice according to a modification of the embodiment.

FIG. 10 is a block diagram showing a first example functionalconfiguration of a measuring projection device according to theembodiment.

FIG. 11 is a time chart showing a first example operation of aprojection system according to the embodiment.

FIG. 12 is a block diagram showing a second example functionalconfiguration of a measuring projection device according to theembodiment.

FIG. 13 is a time chart showing a second example operation of aprojection system according to the embodiment.

FIG. 14 is a diagram showing a third example configuration of aprojection system according to the embodiment.

FIG. 15 is a time chart showing a third example operation of theprojection system according to the embodiment.

FIG. 16 is a diagram showing a fourth example configuration of aprojection system according to the embodiment.

FIG. 17 is a block diagram showing a functional configuration of theprojection adjustment device according to the embodiment.

FIG. 18A is a diagram showing a fourth example configuration of aprojection system according to the embodiment.

FIG. 18B is a time chart showing an example of measurement patternprojection times in the fourth example projection system according tothe embodiment.

FIG. 19A is a diagram showing an example set of projection ranges in thefourth example projection system according to the embodiment.

FIG. 19B is a time chart showing a measurement pattern projectionoperation in the fourth example projection system according to theembodiment.

FIG. 20 is a flowchart showing the procedure of a projection adjustmentmethod of the projection adjustment device according to the embodiment.

FIG. 21A is a diagram showing a first example image display ofprojection ranges of a plurality of measuring projection devices in aprojection adjustment device according to the embodiment.

FIG. 21B is a diagram showing a first example graphical display of theprojection ranges of the plurality of measuring projection devices inthe projection adjustment device according to the embodiment.

FIG. 22A is a diagram showing a second example image display ofprojection ranges of a plurality of measuring projection devices in theprojection adjustment device according to the embodiment.

FIG. 22B is a diagram showing a second example graphical display of theprojection ranges of the plurality of measuring projection devices inthe projection adjustment device according to the embodiment.

FIG. 23A is a diagram showing a third example image display ofprojection ranges of a plurality of measuring projection devices in theprojection adjustment device according to the embodiment.

FIG. 23B is a diagram showing a third example graphical display of theprojection ranges of the plurality of measuring projection devices inthe projection adjustment device according to the embodiment.

FIG. 24 is a diagram showing a fourth example image display ofprojection ranges of a plurality of measuring projection devices in theprojection adjustment device according to the embodiment.

FIG. 25 is a diagram showing a fifth example image display of projectionranges of a plurality of measuring projection devices in the projectionadjustment device according to the embodiment.

DESCRIPTION OF EMBODIMENTS

[Introduction to Content of Embodiment]

In projecting a video content onto a projection target in, for example,projection mapping, it is required to project the video content on theprojection target by positioning the former with respect to the latterin an intended manner. Finally, it is necessary to obtain geometricalpositional information of a target object in the coordinate system of aprojection device.

To perform a projection onto a static target object, it suffices toperform a measurement only once before the projection. In this case,interference between the projection and the measurement can bedisregarded. On the other hand, consider a case of performing anerrorless projection in real time on the basis of a result of a 3Dmeasurement that is performed concurrently on a target object movingand/or deforming dynamically. In this case, it is required to perform ameasurement so as not to affect a video content being projected.

However, the above-mentioned Patent Literature 1 merely discloses thatprojection of a pattern image for 3D measurement with invisible lightmakes it possible to perform a measurement without being affected byvisible light emitted from a visible light source installed at anotherplace. Only a measurement result in a coordinate system that is similarto the coordinate system of an image capture device is obtained by thetechnique of Patent Literature 1.

In the field of measurement, systems disclosed in Referential Non-PatentLiterature 1 and Referential Patent Literature 3, for example, are knownin addition to the system disclosed in the above-mentioned PatentLiterature 1.

Referential Patent Literature 3: JP-A-2013-192189

Referential Non-Patent Literature 1: “Development of a 3,000-fps 3DImaging System Using a High-Speed Projector,” Proceedings of the JSMEConference on Robotics and Mechatronics 2007, “1P1-M02(1)”-“1P1-M02(4),”May 11, 2007

Referential Non-Patent Literature 1 discloses a technique for measuringa 3D shape at high speed using optical pattern projection. A measuringsystem of Referential Non-Patent Literature 1 is equipped with an imagecapture device and a projection device having a light source, a lens,and a mirror element or liquid crystal element. The image capture devicehas a function of performing high-speed imaging. For example, the imagecapture device can perform high-speed imaging at 6,000 fps. Theprojection device can project binary patterns having 1,024×768 pixels ata rate of 6,000 fps or more.

Referential Patent Literature 3 discloses a measuring system whichadjusts a video content on the basis of data taken by imaging. Themeasuring system of Referential Patent Literature 3 is equipped with animage capture device, a projection device, and a computing device. Thecomputing device performs image recognition on a projection target onthe basis of an imaging result of the image capture device. Thecomputing device generates a video of a video content so that the videocontent is projected in an area obtained by recognizing the projectiontarget. The projection device projects the video content onto theprojection target.

The above Referential Non-Patent Literature 1 merely discloses thetechnical level for performing a 3D measurement at high speed.Conventionally, high-speed 3D measurement of a moving object isdifficult because sending of coordinates information of a moving objectrequires images several tens of frames. The technique of ReferentialNon-Patent Literature 1 is meaningful in that it suggests that ahigh-speed measurement is possible.

However, Referential Non-Patent Literature 1 merely discloses a sole 3Dmeasurement technique and refers to nothing about the coordinate systemof the projection device. Referential Non-Patent Literature 1 refers tooffline processing performed after high-speed imaging, that is,non-real-time processing. In the first place, in computer architecturedevices such as personal computers having, as an assumption, imageprocessing at, for example, 60 Hz, a delay of several tens ofmilliseconds or more occurs at the input and output. As a result, it isdifficult to shoot a moving object while projecting its video onto itand feed back a result of the imaging to the projection.

In the technique of the above Referential Patent Literature 3, thedifference between the positions of the image capture device and theprojection device causes a parallax. However, Referential PatentLiterature 3 has no disclosure about how to solve the parallax problemor increase the operation speed of the system

In view of the above circumstances, the present inventors have conceiveda projection system which is equipped with an invisible light projectiondevice capable of high-speed projection of invisible light such asinfrared light, a visible light projection device capable of high-speedprojection of visible light, and an image capture device capable ofhigh-speed imaging and which can measure a position of a target objectwith high accuracy by performing, at high speed, projection ofmeasurement light using invisible pattern light and imaging with it andproject a visible light video content onto the target object whilepositioning it in an intended manner.

Now assume a projection system in which a plurality of projectiondevices are arranged that performs position measurement of a targetobject by projecting measurement light onto it at high speed. Such aprojection system is required to adjust projection times and projectionranges of the plurality of projection devices and to make it possible toperform position measurement and video projection on a target objectwith high accuracy. The above Patent Literature 2 merely discloses howto enable detection of presence of another projection device andmanipulation of a plurality of projection devices by performing infraredlight communications between the plurality of projection devices.

The above projection system which performs position measurement andvideo projection using a plurality of projection devices has a problemthat where projection ranges of the plurality of projection devicesoverlap with each other, measurement light beams interfere with eachother in the overlap regions, disabling a proper position measurement.

Furthermore, where a plurality of projection devices capable ofhigh-accuracy position measurement are provided, a problem arises that apositional relationship between a plurality of projection ranges of therespective projection devices such as an arrangement of the projectionranges and overlaps between the projection ranges cannot be recognizedeasily.

Each embodiment as a specific disclosure of a configuration according tothe present disclosure will be described in detail by referring to thedrawings when necessary. However, unnecessarily detailed descriptionsmay be avoided. For example, detailed descriptions of already well-knownitems and duplicated descriptions of constituent elements havingsubstantially the same ones already described may be omitted. This is toprevent the following description from becoming unnecessarily redundantand thereby facilitate understanding of those skilled in the art. Thefollowing description and the accompanying drawings are provided toallow those skilled in the art to understand the disclosure thoroughlyand are not intended to restrict the subject matter set forth in theclaims.

EMBODIMENT

First, a description will be made of a projection system, projectiondevice, and projection method according to an embodiment that enableproper position measurement by preventing interference betweenmeasurement light beams by controlling the timing relationship betweenprojection times of invisible light beams for position measurement in aplurality of projection devices.

(Outlines of Measuring Projection Device and Projection System)

FIG. 1 is a diagram for outlining of the configuration and function of ameasuring projection device according to the embodiment. The embodimentis directed to a case of measuring positions of target objects andprojecting a video according to position information of the targetobjects using a measuring projection device 100 shown in FIG. 1 which isa projection device for projecting a video onto the target objects.Assumed here as the target objects are a first target object 105 such asa flat screen, a curved screen, or a wall surface and a second targetobject 106 such as a person located in front of the first target object105. They may be referred to below simply as target objects 105 and 106.It is assumed that the second target object 106 such as a person movesas a whole while moving parts of his or her body by, for example,dancing in front of the first target object 105 such as a screen. Thatis, the shape and the position of each part of the target object 106vary according to its motion. Thus, to project a prescribed videocontent onto the target objects 105 and 106, it is necessary to measurea position of the second target object 106 with respect to the firsttarget object 105 and to acquire accurate position information of thetarget object 106.

The measuring projection device 100 is equipped with an image capturedevice 101 which is an example light receiving unit, an infrared lightprojection device 102 which is an example invisible light projectionunit for projecting infrared light as example invisible measurementlight, and a visible light projection device 104 which is an examplevisible light projection unit for projecting visible light. Themeasuring projection device 100 measures positions of the target objects105 and 106 at high speed by projecting, at high speed, infrared patternlight in which sets of projection coordinates have been coded by theinfrared light projection device 102 and capturing images of the targetobjects 105 and 106 at high speed by the image capture device 101. Thedetails of the position measurement of the target objects will bedescribed later. Then the measuring projection device 100 projects aprescribed video by the visible light projection device 104 on the basisof position information of the target objects 105 and 106 particularlyin a state that it is always positioned with respect to the movingtarget object 106. In the embodiment, it is assumed that as describedlater a projection system is constructed by arranging a plurality ofmeasuring projection devices 100 as appropriate.

Several example uses of a projection system using a plurality ofmeasuring projection devices 100 will now be described. FIG. 2 is adiagram showing a first example use of a projection system according tothe embodiment. The first example use is an example in which a pluralityof (in the illustrate example, three) measuring projection devices 100are arranged side by side and position measurement and video projectionare performed by multi-plane projection onto target objects 105 and 106by setting the projection ranges of the respective measuring projectiondevices 100 so that they overlap with each other and cover a wide area.In this case, a problem may occur that positions of the target objects106 cannot be measured accurately due to interference betweenmeasurement light beams in the overlap regions of the projection ranges.

FIG. 3 is a diagram showing a second example use of a projection systemaccording to the embodiment. The second example use is an example inwhich a plurality of (in the illustrate example, three) measuringprojection devices 100 are arranged side by side and positionmeasurement and video projection are performed outdoors, for example, bystack projection onto target objects 105 and 106 by setting theprojection ranges of the respective measuring projection devices 100 insuch a manner that they are superimposed on each other fully or almostfully to attain high-luminance projection. In this case, the projectionangles of the respective measuring projection devices 100 with respectto the target objects 105 and 106 are different from each other andinterference occurs between measurement light beams in most of theprojection area of the measurement light beams where the projectionranges are superimposed on each other: a problem may occur thatpositions of the target object 106 cannot be measured accurately.

FIG. 4 is a diagram showing a third example use of a projection systemaccording to the embodiment. The third example use is an example inwhich a plurality of (in the illustrate example, three) measuringprojection devices 100 are arranged on a circle so as to surround atarget object 106 and position measurement and video projection areperformed on the target object 106 from the plurality of measuringprojection devices 100 by wrapping projection (360° video projection)onto a target object 106 so that the three-dimensional object can beseen from many angles. In this case, since the projection ranges of therespective measuring projection devices 100 are superimposed on eachother largely, a problem may occur that the position of the target 106cannot be measured accurately due to interference between measurementlight beams.

In the embodiment, the above problems are solved by preventinginterference between measurement light beams by controlling an emissiontiming relationship between measurement light beams of the plurality ofmeasuring projection devices 100 by a timing control unit.

FIG. 5 is a diagram showing a first example configuration of aprojection system according to the embodiment. The projection system ofthe first example is equipped with a plurality of measuring projectiondevices 100A and 100B and performs position measurement and videoprojection on target objects 105 and 106 by the measuring projectiondevices 100A and 100B. In the first example, the plurality of measuringprojection devices 100A and 100B are connected to each other and thefirst measuring projection device 100A functions as a master and theother, second measuring projection devices 100B function as slaves.Having a timing control unit inside, the master measuring projectiondevice 100 takes synchronization by instructing the slave measuringprojection devices 100B on operation timings and thereby controls thetiming relationship between position measurement and video projectionoperations of the respective measuring projection devices of theprojection system. The configuration of the illustrated example has thethree measuring projection devices: the central device is a mastermeasuring projection device P1, the left-hand device is a slavemeasuring projection device P2, and the right-hand device is a slavemeasuring projection device P3.

FIG. 6 is a diagram showing a second example configuration of aprojection system according to the embodiment. The projection system ofthe second example is equipped with slave measuring projection devices100B as a plurality of measuring projection devices and performsposition measurement and video projection on target objects 105 and 106by these measuring projection devices 100B. In the second exampleconfiguration, a timing generator 151 is provided outside as an exampletiming control unit and the plurality of slave measuring projectiondevices 100B are connected to the timing generator 151. Constituted by acomputer or the like provided outside, the timing generator 151synchronizes the plurality of slave measuring projection devices 100B byinstructing them on operation timings and thereby controls the timingrelationship between position measurement and video projectionoperations of the respective measuring projection devices of theprojection system. The configuration of the illustrated example has thethree measuring projection devices and all of a central measuringprojection device P1, a left-hand measuring projection device P2, and aright-hand measuring projection device P3 function as slaves.

(Configuration of Measuring Projection Device)

Next, an example configuration and operation of a measuring projectiondevice will be described in more detail.

FIG. 7 is a diagram showing a rough configuration of a measuringprojection device according to the embodiment. The measuring projectiondevice 100 includes an image capture device 101, an infrared lightprojection device 102, a visible light projection device 104, and acomputing device 103.

In the embodiment, as in Referential Non-Patent Literature 1, the imagecapture device 101 can perform imaging at 6,000 frames/sec. Having alarge-scale transfer bandwidth inside with no buffering, the imagecapture device 101 can output data taken to the computing device 103.Furthermore, the image capture device 101 is sensitive in an infraredrange. Based on these assumptions, an example function and operation ofeach device will be described below.

The infrared light projection device 102, which is an example invisiblelight projection unit, projects pattern light (example measurementlight) representing a pattern image in which sets of projectioncoordinates in a projection coordinate system are coded. In thisspecification, the term “projection coordinate system” means acoordinate system in which to specify coordinates of each pixel of avideo content image as a projection image to be projected by the visiblelight projection device 104. Coordinates that determine each pixel of avideo content image are referred to as “projection coordinates” in theprojection coordinate system. Projection coordinates also correspond tocoordinates of each pixel of a pattern image to be projected by theinfrared light projection device 102.

The infrared light projection device 102 is equipped with a lens opticalsystem 111, an infrared LED light source 112, and a display device 113.The lens optical system 111 may either be a single lens or consist of aplurality of lenses (lens group). For example, the plurality of lensesmay include a zoom lens, a focusing lens, etc.

The infrared LED light source 112 emits infrared light (exampleinvisible light) in the form of pattern light. For example, theinvisible light has wavelengths in an infrared range (approximately 700to 1,000 nm). Although in the embodiment the infrared LED light sourceis employed as a light source of invisible light, a light source thatemits ultraviolet light can also be used.

The display device 113 is, for example, a device in which micromirrorsare arranged on 1,024×768 squares, respectively, and generates a patternimage obtained by coding sets of projection coordinates. The displaydevice 113 can output a video at 30,000 frames/sec in the form of binarypatterns. Instead of the reflection-type optical element, the displaydevice 113 may be either a transmission-type optical element or a liquidcrystal device.

The image capture device 101, which is an example light-receiving unit,generates an image taken of pattern light by capturing a pattern lightimage. The image capture device 101 includes an image sensor, a lensoptical system, etc. For example, an image sensor having 1,024×768pixels may be used so as to correspond to the display device 113. Inthis case, if each pixel has a resolution of 8 bits, the transferbandwidth is about 38 Gbps. It is assumed here that the computing device103 is implemented as, for example, an FPGA (field-programmable gatearray). Taking the current levels of the semiconductor technologies,into consideration, the transfer bandwidth of about 38 Gbps is well in arealizable range.

The image capture device 101 has an imaging coordinate system. In thisspecification, the term “imaging coordinate system” means a coordinatesystem in which to specify coordinates of each pixel of an image takenby the image capture device 101. The coordinates of each pixel of animage taken are referred to as “imaging coordinates” to discriminatethem from “projection coordinates.”

The visible light projection device 104, which is an example visiblelight projection unit, projects video light representing a videocontent. The visible light projection device 104 is equipped with a lensoptical system, a visible light source, and a display device that issimilar to the one employed in the infrared light projection device 102.The visible light projection device 104 emits, as video light, light ina visible range (approximately 380 to 780 nm). From the viewpoint ofsimplification, the visible light source can be a monochrome visible LEDlight source. Alternatively, the visible light projection device 104 maynaturally project a full-color video by employing three light sources ofred, blue, and green three colors. If a color wheel capable of rotatingat a sufficiently high speed is available, a full-color video can beprojected by employing a white light source such as a high-pressuremercury lamp in place of the visible LED light source and attaching thecolor wheel to it on its output side. As a further alternative, a lightsource in which light beams having respective wavelengths can beextracted from light emitted from a high-pressure mercury lamp by adichroic prism or the like can be employed as the visible light source.In these manners, every light source can be employed in this disclosure.

The computing device 103, which is an example computing unit, decodes animage taken into projection coordinates information representing sets ofprojection coordinates corresponding to sets of imaging coordinates inthe imaging coordinate system, converts the projection coordinatesinformation into distance information to a target object using theprojection coordinate system as a reference, and determines the detailsof a video content selectively according to the distance information.

FIG. 8 shows example invisible light measurement patterns employed inthe embodiment. FIG. 8 exemplifies part of coded pattern images(coordinate patterns) corresponding to pattern light. The pattern imagesshown in FIG. 8 are obtained by Gray-coding X coordinates and Ycoordinates of the respective mirrors of the display device 113 having1,024×768 micromirrors and expressing a coding result as ablack-and-white binary image (bit-by-bit processing).

The infrared light projection device 102 can project pattern light ontoa target object 107 (corresponding to target objects 105 and 106) on thebasis of a pattern image of 1,024×768 pixels, for example. The Xcoordinates and the Y coordinates of pixels are both larger than 512 andsmaller than or equal to 1,024. In this case, bit 0 to bit 9 (10 bits)of each X coordinate are Gray-coded. As in each X coordinate, bit 0 tobit 9 (10 bits) of each Y coordinate are Gray-coded. Coordinatesinformation can be coded by assigning a total of 20 bits to each set ofcoordinates (10 bits to each of X and Y coordinates). An example inwhich 20-bit information of image data of 40 frames is coded will bedescribed below.

In FIG. 8, (X9 a) denotes a pattern image corresponding to bit 9 afterGray coding of X coordinates. In the embodiment, since projectioncoordinates are coded by Manchester coding, an inverted pattern image inwhich bit 9 is bit-inverted is also used. In FIG. 8, (X9 b) denotes aninverted pattern image obtained by bit-inverting the image pattern (X9a). Likewise, in FIG. 8, (X8 a) denotes a pattern image corresponding tobit 8 after Gray coding of X coordinates and (X8 b) denotes an invertedpattern image obtained by inverting the image pattern (X8 a). In FIG. 8,(X7 a) denotes a pattern image corresponding to bit 7 after Gray codingof X coordinates and (X7 b) denotes an inverted pattern image obtainedby inverting the image pattern (X7 a).

In FIG. 8, (Y9 a) denotes a pattern image corresponding to bit 9 afterGray coding of Y coordinates. In FIG. 8, (Y9 b) denotes an invertedpattern image obtained by inverting the image pattern (Y9 a). Likewise,in FIG. 8, (Y8 a) denotes a pattern image corresponding to bit 8 afterGray coding of Y coordinates and (Y8 b) denotes an inverted patternimage obtained by inverting the image pattern (Y8 a). In FIG. 8, (Y7 a)denotes a pattern image corresponding to bit 7 after Gray coding of Ycoordinates and (Y7 b) denotes an inverted pattern image obtained byinverting the image pattern (Y7 a).

Although not shown in the figure, pattern images and inverted patternimages exist to a measurable resolution; for example, there existpattern images and inverted pattern images corresponding to bit 6 to bit0 of X coordinates and Y coordinates. The infrared light projectiondevice 102 projects 40 patterns including the above patterns to thetarget object 107 in order. The image capture device 101 shoots theprojected pattern images in order.

FIG. 9 is a diagram showing the configuration of a measuring projectiondevice according to a modification of the embodiment. The measuringprojection device 140 according to the modification, which is an examplein which the projection devices of each of the measuring projectiondevices 100 shown in FIGS. 1 and 7 are integrated together, has aprojection device 142 including an infrared light projection device anda visible light projection device. The measuring projection device 140having such an integration-type projection device 142 may be used.

(Functional Configuration of Measuring Projection Device)

FIG. 10 is a block diagram showing a first example functionalconfiguration of a measuring projection device according to theembodiment. A computing device 103 has a function of controlling theentire measuring projection device 100. For example, the computingdevice 103 can be implemented by a computing device as typified by acomputer and a processor or a semiconductor integrated circuit. Forexample, the semiconductor integrated circuit is an ASIC(application-specific integrated circuit) or an FPGA. The functions ofrespective constituent elements of the computing device 103 may berealized by using a memory in which computer programs for performing thefunctions of the respective constituent elements are installed andcausing a processor incorporated in the semiconductor integrated circuitto run the computer programs successively.

The computing device 103 is equipped with an image input unit 401, apattern decoding unit 402, a frame memory unit 403, a code decodingmemory unit 404, a coordinates conversion unit 405, a coordinatesconversion memory unit 406, a coordinates interpolation unit 407, acontent generation unit 408, a content memory unit 409, an image outputunit 410, and a pattern generation unit 411. Each memory unit in thecomputing device 103 may be constituted by a RAM, for example.

The computing device 103 is also equipped with an externalsynchronization interface (I/F) 152 which is an example timing controlunit. Where a plurality of measuring projection devices 100 arecontrolled by timing control units provided inside the measuringprojection devices, the external synchronization interface 152 of amaster measuring projection device functions as a timing control unitand sends timing signals to the external synchronization interfaces 152of the other, slave measuring projection devices. Where a plurality ofmeasuring projection devices 100 are controlled by a timing control unitprovided outside the measuring projection devices, a timing generator151 provided outside functions as a timing control unit and sends timingsignals to the external synchronization interfaces 152 of the pluralityof slave measuring projection devices.

Where the external synchronization interfaces 152 is connected to thetiming generator 151 or the external synchronization interface 152 ofanother measuring projection device, it may be, for example, acommunication interface connected by wire or a communication interfaceconnected wirelessly. The wired communication interface may be, forexample, USB (Universal Serial Bus) or Ethernet (registered trademark).The wireless communication interface may be, for example, Bluetooth(registered trademark) or wireless LAN. Incidentally, the externalsynchronization interfaces 152 may be composed of a light source and anoptical sensor of, for example, infrared light and control the operationtiming between measuring projection devices by turning on/off of lightor optical communication.

(Operation of Projection System)

FIG. 11 is a time chart showing a first example operation of aprojection system according to the embodiment. In the embodiment, it isassumed that projection of an infrared light measurement pattern by theinfrared light projection device 102, camera exposure by the imagecapture device 101, reading-out and computation, or video projection bythe visible light projection device 104 is performed every prescribedunit time, the prescribed unit time being 1 ms, for example. The term“reading-out and computation” as used here means processing that theimage capture device 101 reads out an image taken and transfers it tothe computing device 103 (referred to below as reading-out and transferprocessing or processing of transfer to the computing device 103) plusprocessing that the computing device 103 calculates position measurementinformation from the transferred image taken. The processing that thecomputing device 103 calculates position measurement information may beperformed either at the same time as (parallel with) the reading-out andtransfer processing performed by the image capture device 101 or aftercompletion of all or part of the reading-out and transfer processingperformed by the image capture device 101 (the processing of transfer tothe computing device 103). Where the above two kinds of processing areperformed at the same time (in parallel) as in the embodiment,calculation of position measurement information can be performed so asto overlap with a readout period by constructing a dedicated computingdevice using an FPGA, for example. Unlike in the embodiment, the timetaken by the reading-out and transfer of an image taken need not alwaysbe the same as the time taken by the position information measurement.The above-described reading-out and transfer and calculation of positioninformation measurement also apply to example operations to be describedwith reference to drawings that follow FIG. 11.

The pattern generation unit 411 turns on the infrared LED light source112 of the infrared light projection device 102 in a measurement patternprojection period. The pattern generation unit 411 generates a patternimage for pattern projection by the above-described method. The patterngeneration unit 411 outputs image data representing the pattern image tothe image output unit 410 so as to allow the display device 113 of theinfrared light projection device 102 to perform pattern projection formeasurement. The image output unit 410 outputs the image data receivedfrom the pattern generation unit 411 and turn-on information for theinfrared LED light source 112 to the infrared light projection device102 and the image input unit 401. Since pattern light of measurementlight representing the pattern image is projected in the form ofinvisible light, it does not influence the visual sense of a humanthough it is subjected to imaging by the image capture device 101 andmeasurement.

The pattern generation unit 411 can output one pattern in 1/6,000 sec.In each measurement pattern projection period, the pattern generationunit 411 outputs a total of 40 frames consisting of 10-bit coordinateimages of X coordinates and Y coordinates and their inverted images. Onthe other hand, the image capture device 101 shoots 40 frames at thesame rate as the frame output rate of the display device 113(synchronized).

The image output unit 410 outputs a pattern image to the infrared lightprojection device 102 in synchronism with the image data output timingof the pattern generation unit 411. The infrared light projection device102 projects a pattern image onto a target object. The image input unit401 controls exposure of the image capture device 101 in synchronismwith the pattern image output timing of the image output unit 410.Controlled in this manner, the image capture device 101 shoots a40-frame pattern image in a camera exposure period.

The image input unit 401 receives the pattern image taken (data taken)by the image capture device 101 in a reading-out and computation period.The image input unit 401 sends the received data taken to the patterndecoding unit 402. The image input unit 401 determines a patterncorresponding to the received data taken in synchronism with the imageoutput unit 410.

The pattern decoding unit 402 decodes the image taken representing thepattern image received from the image capture device 101 into projectioncoordinates information representing sets of projection coordinatescorresponding to sets of imaging coordinates in the imaging coordinatesystem in the reading-out and computation period. The function of thepattern decoding unit 402 will be described below in more detail.

If the data taken received from the image input unit 401 represents an Xcoordinate/Y coordinate non-bit-inversion image, the pattern decodingunit 402 writes that data taken to the frame memory unit 403. If thedata taken represents an X coordinate/Y coordinate bit-inversion image,the pattern decoding unit 402 calculates the difference between thatimage and a non-bit-inversion image recorded in the frame memory unit403 in advance while reading out the latter. By taking the differencebetween a non-bit-inversion image and a bit-inversion image in thismanner, discrimination between 0s and 1s of projection light is madepossible without depending on the color of the projection target orambient light. By judging a region where the difference is smaller thanor equal to a prescribed value to be a region in which the projectionlight is not projected, that region can be eliminated from a measurementtarget region.

The code decoding memory unit 404 is provided with a writing region foreach pixel of the image capture device 101. After taking the differencebetween a non-bit-inversion image and a bit-inversion image, the patterndecoding unit 402 writes individual bit values of Gray-coded coordinatesdata in writing regions bit by bit. This manipulation of writingcoordinates data of 40 frames is performed during an exposure time ofthe image capture device 101. As a result, information that correspondsto each pixel of the image capture device 101 and indicates whether an Xcoordinate and a Y coordinate of the infrared light projection device102 exist and 10-bit values representing the X coordinate and the Ycoordinate (if they exist), respectively, are written to the codedecoding memory unit 404. Finally, the pattern decoding unit 402re-converts the Gray-coded coordinates data recorded in the codedecoding memory unit 404 into binary data and outputs the latter to thecoordinates conversion unit 405.

As a result of the pieces of processing performed so far, one can knowfrom what pixel of the infrared light projection device 102 projectionlight that has reached a certain pixel position of the image capturedevice 101 has been projected. That is, one can know correspondingrelationships between sets of projection coordinates of the infraredlight projection device 102, that is, sets of projection coordinates inthe projection coordinate system of the visible light projection device104, and sets of imaging coordinates in the imaging coordinate system ofthe image capture device 101. Thus, if a positional relationship betweenthe image capture device 101 and the infrared light projection device102 is known, a distance to the target object can be obtained for eachpixel in an image by triangulation. However, information thus obtainedis distance information corresponding to each pixel in an image capturedby the image capture device 101. Thus, in the embodiment, distanceinformation of imaging coordinates corresponding to each pixel in animage captured by the image capture device 101 is converted intodistance information corresponding to pixel coordinates of the infraredlight projection device 102, that is, distance information of projectioncoordinates of the visible light projection device 104.

The coordinates conversion unit 405 writes data received from thepattern decoding unit 402 in a region, determined by addressescorresponding to sets of projection coordinates of the visible lightprojection device 104, of the coordinates conversion memory unit 406.Then the coordinates conversion unit 405 generates distance informationcorresponding to projection coordinates of the visible light projectiondevice 104 by reading out each piece of distance information (an Xcoordinate and a Y coordinate of the visible light projection device 104are read out in this order).

In doing so, a projection pixel having no corresponding point may occur.More specifically, light rays corresponding to a plurality of certainpixels of a pattern image projected onto the target object may bedetected by one pixel in an image captured by the image capture device101. In this case, from the characteristic of Gray codes, a projectionpixel having no corresponding point is incorporated into one of twoadjacent projection pixels and hence the other projection pixel comesnot to have a corresponding point.

The coordinates interpolation unit 407 receives distance informationcorresponding to each set of projection coordinates of the visible lightprojection device 104 from the coordinates conversion unit 405. Thecoordinates interpolation unit 407 interpolates distance information ateach set of projection coordinates having no distance information. Thisis done only at a position around which a certain number of sets ofprojection coordinates having distance information that enablesinterpolation exist, by an interpolation method such as linearinterpolation using pieces of distance information of nearby sets ofcoordinates. The coordinates interpolation unit 407 outputs each pieceof distance information calculated on the basis of pieces of projectioncoordinates to the content generation unit 408. As described above, ahigh-speed real-time position measuring operation is enabled byperforming reading-out of an image taken of a pattern image andcalculation of position information including information of distancesto the target object.

The content generation unit 408 generates a video content for projectionin a video projection period. The content generation unit 408 processesa video content recorded in the content memory unit 409 in advance onthe basis of the distance information received from the coordinatesinterpolation unit 407 and outputs the processed video content to theimage output unit 410. In the following, a term “processed videocontent” may be used to discriminate it from the unprocessed videocontent recorded in advance.

The content generation unit 408 generates a video content that is freeof a coordinate deviation and corresponds to distances to the targetobject accurately. Furthermore, the content generation unit 408 candetermine the details of a video content selectively according todistance information. For example, the content generation unit 408 canperform processing of cutting out and detecting only an object locatedat a certain distance and draw a video content for visible lightprojection accurately. The content generation unit 408 outputs aprocessed video content for projection to the image output unit 410.

The image output unit 410 outputs the video content for visible lightprojection generated in the video projection period to the visible lightprojection device 104. The visible light projection device 104 turns onthe visible light source and projects video light corresponding to thevideo content. The display device of the visible light projection device104 can output 30,000 binary frames per second. Thus, for example, a256-gradation image can be projected using 255 frames in 8.5 ms. Sincethis projection is performed by the visible light source, the projectedimage can be recognized visually by a human. In the above-describedmanner, position measurement and projection can be performedcontinuously.

In the measuring projection device 100 according to the embodiment,since video projection and position measurement are performed by thesame measuring projection device, occurrence of a deviation between aprojection and a measurement can be suppressed in principle and ageometrical measurement can be performed in a superimposed mannerwithout interfering with a visible light image. Furthermore, if thecomputing device 103 can decode a pattern image taken by the imagecapture device 101, the measuring projection device 100 can be used inpractice even if the accuracy of its installation is not sufficientlyhigh. This secures easy installation. Furthermore, high robustness canbe obtained against increase of aging errors relating to installation.

Now, how a plurality of measuring projection devices operate in aprojection system according to the embodiment will be described withreference to FIG. 11. In the embodiment, a timing control unit 150 suchas the external synchronization interface 152 of a master measuringprojection device or the external timing generator 151 generates andoutputs timing signals, whereby an operation timing relationship betweenthe plurality of measuring projection devices 100 is controlled. Theexternal synchronization interface 152 of each measuring projectiondevice receives the timing signal and inputs it to the image output unit410, and output of a pattern image from the image output unit 410 andinput of an image taken to the image input unit 401 are controlledaccording to the timing signal. The plurality of measuring projectiondevices are synchronized with each other in this manner.

FIG. 11 shows an example operation of a case that three measuringprojection devices P1, P2, and P3 perform position measurements by turnsin a time-divisional manner. In this example operation, the number ofphases is three which is equal to the number of measuring projectiondevices. The plurality of measuring projection devices P1, P2, and P3perform infrared light measurement pattern projection and cameraexposure in order (by turns) according to a timing signal that is outputfrom the timing control unit 150 every prescribed time. After completionof measurement pattern projection and camera exposure of the measuringprojection device P1, measurement pattern projection and camera exposureof the measuring projection device P2 are performed and measurementpattern projection and camera exposure of the measuring projectiondevice P3 follow. The unit time shown in FIG. 11 that is the intervalbetween timing signals is 1 ms, for example. Thus, measurement patternprojection and camera exposure can be performed 1,000 times per second(1,000 fps) in order by the plurality of measuring projection devices.

After performing measurement pattern projection and camera exposure byturns in a time-divisional manner, the measuring projection devices P1,P2, and P3 perform reading-out of an image taken of a pattern image andcalculation of position information including information of a distanceto a target object at timings of the next unit times. Then the measuringprojection devices P1, P2, and P3 generate video contents according torespective pieces of measured position information of the target objectat timings of the next unit times and project videos. In FIG. 11,numbers “1,” “2,” and “3” shown in respective blocks indicatingoperation timings are example frame numbers; in the illustrate example,video projection of one frame is performed every three unit times (3ms). Although timing deviations occur between sets of video projectionframes of the plurality of measuring projection devices P1, P2, and P3,this does not affect the visual sense of a human because the deviationsare as short as a small multiple of 1 ms.

In the above operation, each of the measuring projection devices P1, P2,and P3 projects measurement light at a projection timing that isdifferent than the other measuring projection devices do. Since theprojection range of the measuring projection device P1 overlaps with theprojection ranges of the measuring projection devices P2 and P3, themeasuring projection device P1 performs measurement light projection ata projection timing that is different from projection timings of themeasuring projection devices P2 and P3. The projection range of themeasuring projection device P2 overlaps with that of the measuringprojection device P1 but does not overlap with that of the measuringprojection device P3. Thus, the measuring projection devices P2 and P3may perform measurement light projection at the same projection timing.The measuring projection device P2 performs measurement light projectionat such a timing that the measuring projection device P1 performsreading-out of an image taken and calculation of position information(computation timing). The measuring projection device P2 performsreading-out of an image taken and calculation of position information atsuch a timing that the measuring projection device P1 performsmeasurement light projection. Furthermore, the measuring projectiondevice P1 performs measurement light projection at such a timing that itperforms projection of a video content.

As described above, in the embodiment, position measurement is performedon a target object in such a manner that the plurality of measuringprojection devices perform projection of measurement light and exposureby turns while the timing control unit controls them. This operationmakes it possible to prevent interference between measurement lightbeams of the plurality of measuring projection devices. In particular,adjacent measuring projection devices are prevented from projectingmeasurement light at the same time, whereby interference betweenmeasurement light beams in an overlap region of projection ranges can beprevented. Thus, each measuring projection device of the projectionsystem can perform a proper position measurement. For example, whenposition measurement and video projection are performed on a movingtarget object such as a dancer using a plurality of measuring projectiondevices, video projection can be performed repeatedly while positionmeasurement is performed accurately in real time. Furthermore, wherethere exist a fixed target object such as a screen and a moving targetobject such as a dancer, even a projection system using a plurality ofmeasuring projection devices can measure positions of the target objectsaccurately in real time and generate and project video contentsindividually so that they are suitable for the positions of the targetobjects, respectively.

FIG. 12 is a block diagram showing a second example functionalconfiguration of a measuring projection device according to theembodiment. The second example functional configuration of a measuringprojection device is an example configuration that is obtained bychanging part of the above-described first example configuration of ameasuring projection device and adding a function. A computing device123 employed in the embodiment has a setting input unit 131 to whichsetting information is input. The other part of the configuration andthe other functions of the computing device 123 are the same as those ofthe computing device 103 of the first example, and hence only featuresthat are different than in the first example will be described here.

The setting input unit 131 receives setting information to be used forsetting a measurement light emission time division number and a lightemission quantity of measurement light according to, for example, thenumber of phases of measuring projection devices that operate in orderor the number of measuring projection devices that are usedsimultaneously. The setting information may be generated by anexternally provided control device such as a computer and transmitted tothe setting input unit 131 of the measuring projection device. Thesetting input unit 131 sends the received setting information to theimage output unit 410 and the infrared light projection device 102 andthus serves to set a light emission time and a light emission quantityof measurement light.

In the embodiment, where n measuring projection devices perform positionmeasurement by turns, a time period for which measurement patternprojection and camera exposure are performed is set at 1/n of a unittime. To compensate for the shortening of the measurement patternprojection periods to 1/n of the unit time, the light emission quantityof invisible measurement light is increased. For example, the drivecurrent that flows when the infrared LED light source emits light isincreased by a factor of n. This may be done by changing the peak powerof the infrared LED light source when it emits light, according to alight emission time. To attain this purpose, the drive power can becontrolled by various methods other than changing the drive current ofthe infrared LED light source, that is, by changing the drive voltage,the PWM ratio, or the like. Setting information including a divisionaltime of measurement pattern projection and camera exposure and anincreased drive current is sent from the setting input unit 131 to theimage output unit 410 and the infrared light projection device 102 andto the image input unit 401 via the image output unit 410.

Where the measuring projection device performs, at high speed, a seriesof pieces of processing of measurement pattern projection, cameraexposure, reading-out and transfer of an image taken, computation ofposition information measurement, and video projection, it is highlyprobable that the processing of reading-out and transfer of an imagetaken becomes a rate determining step (i.e., bottleneck). Thus, toperform this series of pieces of processing at high speed, it ispreferable that each processing time be made variable with thereading-out and transfer time made a reference (unit time). For example,the period of measurement pattern projection (and camera exposure) maybe made 1/n of the reading-out and transfer time (unit time). It ispreferable that the unit time be equal to a time that is taken toperform reading-out and transfer when the image capture device and/orthe computing device has operated to show its full ability. Where asmentioned above the unit time is determined on the basis of theperformance of the image capture device and/or the computing device, theunit time may be stored in a memory provided inside the measuringprojection device in advance, transmitted from outside the measuringprojection device over a desired network, or measured by conducting atest actually in the measuring projection device.

FIG. 13 is a time chart showing a second example operation of aprojection system according to the embodiment. FIG. 13 shows an exampleoperation of a case that three measuring projection devices P1, P2, andP3 perform position measurements by turns in a time-divisional manner.In this case, the period of measurement pattern projection and cameraexposure is set as short as ⅓ (e.g., ⅓ ms) of the ordinary unit time andthe drive current (drive power) of the infrared LED light source isincreased by a factor of 3. With this measure, the emission lightquantity of the infrared LED light source is increased by a factor of 2to 3, for example. In LED light sources etc., the drive current ratingis determined by the average current per unit time. Thus, in the case ofintermittent lighting having divisional light emission times, themaximum current can be increased while the average current in the unittime is kept the same. That is, the maximum light emission quantity canbe increased without changing the power consumption. Incidentally, thenumber of phases in this example operation is three which is equal tothe number of measuring projection devices.

The plurality of measuring projection devices P1, P2, and P3 performinfrared light measurement pattern projection and camera exposure inorder (by turns) in ⅓ of a prescribed unit time according to timingsignals that are output from the timing control unit 150. In theillustrated example, the measuring projection devices P1, P2, and P3perform measurement pattern projection and camera exposure by turns inorder of P1→P2→P3 in one unit time. Upon completion of measurementpattern projection and camera exposure, each of the measuring projectiondevices P1, P2, and P3 performs reading of an image taken of a patternimage and calculation of position information of a target object in oneunit time and then performs generation of a video content and videoprojection in one unit time according to the position information of thetarget object. In this case, the time period for which infrared lightmeasurement pattern projection and camera exposure are performed isshorter than the time period for which reading of an image taken andcalculation of position information are performed.

In the embodiment, since the time period for which measurement patternprojection and camera exposure are performed is shortened by timedivision, projection using the plurality of measuring projection devicescan be performed while the frame rate of position measurement and videoprojection is kept the same. Furthermore, the accuracy of positionmeasurement can be increased by increasing the emission light quantityof measurement light.

FIG. 14 is a third example configuration of a projection systemaccording to the embodiment. The third example configuration of aprojection system is an example configuration in which the configurationof the timing control unit of each measuring projection device ischanged. The projection system according to the embodiment is equippedwith a plurality of measuring projection devices which are a mastermeasuring projection device 160A and slave measuring projection devices160B. Each slave measuring projection device 160B is equipped with anoptical sensor 153 for detecting invisible measurement light. Like theexternal synchronization interface 152 of each measuring projectiondevice of the first example, the optical sensor 153 functions as atiming control unit. The other part of the configuration and the otherfunctions are the same as in the above-described first example andsecond example, and hence only different features will be describedhere.

FIG. 15 is a time chart showing a third example operation of theprojection system according to the embodiment. FIG. 15 shows an exampleoperation of a case that the three measuring projection devices P1, P2,and P3 perform position measurement by turns in a time-divisionalmanner. In this example, the central measuring projection device P2functions as a master and the left-hand measuring projection device P1and the right-hand measuring projection device P3 function as slaves. Inthis example operation, the number of phases is two which is smallerthan the number of measuring projection devices.

First, the master measuring projection device P2 performs measurementpattern projection and camera exposure. In this period, since theprojection ranges of adjacent measuring projection devices overlap witheach other, in each of the slave measuring projection devices P1 and P3invisible measurement light is received by the optical sensor 153 andprojection of master measurement light is thereby detected. Each of theslave measuring projection devices P1 and P3 receives, by the opticalsensor 153, as a timing signal, invisible measurement light projected bythe master measuring projection device P2.

Upon completion of the projection of measurement light by the mastermeasuring projection device P2, each of the slave measuring projectiondevices P1 and P3 detects the stop of emission of measurement light bythe optical sensor 153 and inputs, to the image output unit 410, atiming signal to serve as a trigger for a start of projection of ameasurement pattern. In response, each of the slave measuring projectiondevices P1 and P3 performs measurement pattern projection and cameraexposure. In the illustrated example, as in the second example operationof a projection system shown in FIG. 13, the emission time ofmeasurement light is divided and the emission light quantity ofmeasurement light is increased. While each of the slave measuringprojection devices P1 and P3 is projecting measurement light, theoptical sensor 153 detects the measurement light projected by the selfdevice. For example, each of the slave measuring projection devices P1and P3 may detect only measurement light projected by the mastermeasuring projection device P2 by, for example, masking an output of theoptical sensor 153 during measurement pattern projection of the selfdevice.

In the above operation, since the projection range of the measuringprojection device P2 overlaps with that of each of the measuringprojection devices P1 and P3, the measuring projection device P2projects measurement light at a projection timing that is different thanthe measuring projection devices P1 and P3 do. Whereas the projectionrange of the measuring projection device P1 overlaps with that of themeasuring projection device P2, it does not overlap with that of themeasuring projection device P3. Thus, the measuring projection devicesP1 and P3 project measurement light at the same projection timing. Withthis measure, the number of phases can be made smaller than the numberof measuring projection devices. That is, target objects can be measuredefficiently in an even shorter time while interference betweenmeasurement light beams is prevented.

Upon completion of measurement pattern projection and camera exposure byitself, each of the measuring projection devices P1, P2, and P3 performsreading-out of an image taken of a pattern image and calculation ofposition information of the target objects in one unit time and thenperforms generation of a video content and video projection in one unittime according to the position information of the target objects.

In the embodiment, since the timing control units are provided which areoptical sensors, a timing control for measurement light beams of theplurality of measuring projection devices can be performed withoutproviding communication units such as external synchronizationinterfaces or the like.

As described above, the projection system according to the embodiment isa projection system including the plurality of measuring projectiondevices 100 as projection devices which perform position measurement andprojection on target objects 105 and 106. Each of the measuringprojection devices 100 is equipped with the invisible light projectiondevice 102 which projects invisible measurement light onto the targetobjects, the image capture device 101 which receives reflection light,reflected from the target objects, of the measurement light, thecomputing device 103 which calculates position information of the targetobjects on the basis of the reflection light of the measurement light,and the visible light projection device 104 which projects a visiblelight video content on the basis of the position information of thetarget object. A first measuring projection device and a secondmeasuring projection device of the plurality of measuring projectiondevices 100 project the measurement light at different projectiontimings.

With this configuration, interference between measurement light beamscan be prevented in the plurality of projection devices. Even in a casethat, for example, the projection ranges of adjacent projection devicesoverlap with each other, interference between measurement light beamscan be prevented by setting different projection timings for respectivemeasurement light beams, whereby proper position measurement is enabled.Thus, measurement of target objects and video projection can beperformed properly in the projection system employing the plurality ofprojection devices.

In the projection system, the infrared light projection device 102 ofthe second measuring projection device projects the measurement lightwhile the computing device 103 of the first measuring projection deviceperforms a computation relating to the position information. With thismeasure, target objects can be measured efficiently in a short timewhile interference between measurement light beams is prevented.

In the projection system, the computing device 103 of the secondmeasuring projection device performs a computation relating to theposition information and calculates a shape and a position of the targetobjects while the visible light projection device 104 of the firstmeasuring projection device projects the video content. With thismeasure, target objects can be measured efficiently in a short timewhile interference between measurement light beams is prevented.

In the projection system, the infrared light projection device 102 ofthe first measuring projection device projects the measurement lightwhile the visible light projection device 104 of the first measuringprojection device projects the video content. With this measure,measurement of target objects and video projection can be performedefficiently in a short time while interference between measurement lightbeams is prevented.

In the projection system, the infrared light projection device 102 ofthe second measuring projection device projects the measurement lightwhile information relating to the reflection light received by the imagecapture device 101 is transferred to the computing device 103. With thismeasure, target objects can be measured efficiently in a short time.

In the projection system, in each of the measuring projection devices100 a time period for which the infrared measurement light is projectedis shorter than a time period for which information relating to thereflection light received by the image capture device 101 is transferredto the computing device 103. For example, the measuring light is emittedin such a short time as 1/(the number of the plurality of measuringprojection devices in the projection system) of the time period forwhich information relating to the reflection light received by the imagecapture device 101 is transferred to the computing device 103. Thismeasure enables position measurement of target objects in a short time,whereby target objects can be measured efficiently in a short time.Furthermore, the emission light quantity can be increased as theemission time of measurement light is shortened. This contributes toincrease of the accuracy of the position measurement.

In the projection system, in each of the measuring projection devices100 the time period for which the measurement light is projected is setaccording to at least one of the number of measuring projection devicesin the projection system, the number of measuring projection devicesthat are used at the same time in the projection system, and the numberof phases of measuring projection devices that operate in order in theprojection system. For example, where three measuring projection devicesare used, the time period for which the measurement light is projectedis set at ⅓ of an ordinary computation time and an ordinary projectiontime. The light quantity of the measurement light is set at, forexample, three times an ordinary value. By shortening the projectiontime of the measurement light by dividing it in this manner, positionsof target objects can be measured in a short time, which makes itpossible to perform projection using a plurality of measuring projectiondevices while the frame rate of position measurement and videoprojection is kept the same. Furthermore, since the emission lightquantity of the measurement light is increased, the accuracy of positionmeasurement can be increased.

In the projection system, in each of the measuring projection devices100 the drive power of a light source of the infrared light projectiondevice 102 is adjusted according to the set time period for which theinvisible measurement light is projected. For example, the peak power oflight emission of an infrared LED light source is changed according tothe emission time of the invisible measurement light, for example, thenumber of phases of the measuring projection devices in the projectionsystem. With this measure, the maximum power and the maximum lightemission quantity can be increased in a state that the average power ina unit time of light emission of measurement light is kept the same,whereby the accuracy of position measurement can be increased.

In the projection system, the timing control unit 150 which controls theprojection timings is provided and each of the measuring projectiondevices 100 receives a timing signal from the timing control unit 150and projects the measurement light at a prescribed timing that is inaccordance with the timing signal. For example, the timing generator 151provided outside the devices, the external synchronization interfaces152 provided in the respective devices, or the optical sensors 153 fordetecting invisible light are used as the timing control unit 150. Withthis measure, the projection timings of measurement light can becontrolled and synchronized with each other properly in the plurality ofmeasuring projection devices 100.

In the projection system, each of the first measuring projection deviceand the second measuring projection device functions as, for example, aslave, receives a timing signal from the timing control unit 150 whichcontrols the projection timings, and projects the measurement light at aprescribed timing that is in accordance with the timing signal. Withthis measure, the projection timings of measurement light can becontrolled and synchronized with each other properly in the plurality ofmeasuring projection devices 100.

In the projection system, the second measuring projection devicefunctions as, for example, a slave, receives a timing signal from thefirst measuring projection device which functions as a master, andprojects the measurement light at a prescribed timing that is inaccordance with the received timing signal. With this measure, theprojection timings of measurement light can be controlled andsynchronized with each other properly in the plurality of measuringprojection devices 100.

In the projection system, the second measuring projection device has theoptical sensor 153 which detects invisible light, and receives theinvisible measurement light projected by the first measuring projectiondevice by the optical sensor 153 as the timing signal. By using, in thismanner, as a timing signal, measurement light projected by anothermeasuring projection device, the projection timings of measurement lightcan be controlled and synchronized with each other properly.

In the projection system, a third measuring projection device theprojection range of the measurement light of which overlaps with theprojection range of the measurement light of each of the first measuringprojection device and the second measuring projection device projectsthe measurement light at a different projection timing than each of thefirst measuring projection device and the second measuring projectiondevice does. With this measure, interference between measurement lightbeams can be prevented in a plurality of measuring projection devicesand measurement of target objects and video projection can be performedproperly in a projection system employing a plurality of projectiondevices.

In the projection system, a third measuring projection device theprojection range of the measurement light of which overlaps with theprojection range of the measurement light of the first measuringprojection device but does not overlap with the projection range of themeasurement light of the second measuring projection device projects themeasurement light at the same projection timing as the second measuringprojection device does. With this measure, target objects can bemeasured efficiently in a short time while interference betweenmeasurement light beams can be prevented in a plurality of measuringprojection devices.

The projection device according to the embodiment is the measuringprojection device 100 in the projection system including the pluralityof projection devices which perform position measurement and projectionon target objects 105 and 106. The measuring projection device 100 isequipped with the infrared light projection unit 102 which projectsinfrared measurement light (invisible light) onto the target objects,the image capture unit 101 which receives reflection light, reflectedfrom the target objects, of the measurement light and performs captureof image using it, the computing device 103 which calculates positioninformation of the target objects on the basis of the reflection lightof the measurement light; and the visible light projection device 104which projects a visible light video content on the basis of theposition information of the target objects. The measuring projectiondevice 100 projects the measurement light at a different projectiontiming than the other measuring projection devices do. With thisconfiguration, measurement of target objects and video projection can beperformed properly in a projection system employing a plurality ofprojection devices.

The projection device receives a timing signal from another, firstmeasuring projection device and projects the measurement light at aprescribed timing that is in accordance with the timing signal. Withthis measure, the projection timing of measurement light can becontrolled properly in the plurality of measuring projection devices100, whereby synchronization with the other measuring projection devicesis made and interference between measurement light beams can beprevented.

The projection device transmits a timing signal to another, secondmeasuring projection device and causes it to project measurement lightat a prescribed timing that is in accordance with the timing signal.With this measure, the projection timing of measurement light can becontrolled properly in the plurality of measuring projection devices100, whereby synchronization with the other measuring projection devices100 is made and interference between measurement light beams can beprevented.

The projection method according to the embodiment is a projection methodin the projection system including the plurality of measuring projectiondevices 100 as projection devices which perform position measurement andprojection on target objects 105 and 106. Each of the projection devices100 projects infrared measurement light (invisible light) onto thetarget objects by the infrared light projection device 102, receivesreflection light, reflected from the target objects, of the measurementlight and performs capture of image using it by the image capture device101, calculates position information of the target objects on the basisof the reflection light of the measurement light by the computing device103, and projects a visible light video content on the basis of theposition information of the target objects by the visible lightprojection device 104. A first measuring projection device and a secondmeasuring projection device of the plurality of measuring projectiondevices 100 project the measurement light at different projectiontimings. With this method, measurement of target objects and videoprojection can be performed in a projection system employing a pluralityof projection devices.

Next, a projection adjusting program and a projection adjustment methodthat make it possible to recognize a positional relationship between aplurality of projection ranges in adjusting projection operation itemssuch as projection times and projection ranges in a projection systemhaving a plurality of projection devices will be described as anotherembodiment.

(Configurations of Projection System and Projection Adjustment Device)

FIG. 16 is a diagram showing a fourth example configuration of aprojection system according to this embodiment. This embodiment isdirected a case that in the projection system shown in FIG. 16projection operations of a plurality of measuring projection devices 100are adjusted or a user's work of adjusting projection operations isassisted using a projection adjustment device 200 constituted by apersonal computer (PC) or the like. In this example, a case is assumedthat the projection ranges of the plurality of respective measuringprojection devices 100 are set so as to overlap with each other and tocover a wide area when multi-plane projection is performed on targetobjects 105 and 106 by the plurality of measuring projection devices100.

Connected to a monitor 250 having a display for information display, theprojection adjustment device 200 displays, on the monitor 250, a displaypicture including various kinds of projection information for adjustmentof projection operations. Constituted by an information processingdevice such as a PC having a processor and a memory, the projectionadjustment device 200 realizes such functions as display of projectioninformation and automatic adjustment of projection operations by runninga prescribed computer program.

FIG. 17 is a block diagram showing a functional configuration of theprojection adjustment device according to the embodiment. The projectionadjustment device 200 is equipped with a processing unit 210, a storageunit 220, and a communication interface (I/F) 230. Connected to themeasuring projection devices 100 via the communication interface 230,the projection adjustment device 200 transmits and receives variouskinds of information such as setting information relating to ameasurement operation, projection range information, and positionmeasurement information of target objects. Connected to a display unit240 and an input unit 260, the projection adjustment device 200 displaysa display picture on the display unit 240 and receives a manipulationinstruction from the input unit 260.

The storage unit 220 has a memory device including at least one of asemiconductor memory such as a flash memory, a storage device such as anSSD (solid-state drive) or an HDD (hard disk drive), etc. The storageunit 220 is stored with a projection adjustment program 221 forrealizing functions relating to adjustment of projection operations.

The processing unit 210 has a processor such as a CPU (centralprocessing unit) or a DSP (digital signal processor). The processingunit 210 executes a process according to the projection adjustmentprogram 221 and thereby realizes such functions as positionalrelationship display 211 and projection time setting 212.

The communication interface 230 is an interface that transmits andreceives information to and from external devices such as the measuringprojection devices 100 by a wired communication or a wirelesscommunication. USB (Universal Serial Bus), Ethernet (registeredtrademark), etc. may be used as a wired communication interface.Bluetooth (registered trademark), wireless LAN, etc. may be used as awireless communication interface.

The projection adjustment device 200 displays, as a function of thepositional relationship display 211, projection position informationsuch as an arrangement of projection ranges of the plurality ofmeasuring projection devices, manners of overlap between the projectionranges, and a connection relationship between the projection ranges inany of various display forms such as figure display, text display, icondisplay, and graphic display. The projection adjustment device 200makes, as a function of the projection time setting 212, projectiontime-related settings such as a projection timing relationship betweenthe plurality of measuring projection devices and projection order orphases on the basis of the projection position information such as thearrangement of projection ranges of the respective measuring projectiondevices and the connection relationship between the measuring projectiondevices.

(Operation of Projection System)

A description will now be made of setting of projection times and anarrangement of projection ranges in a projection system having aplurality of measuring projection devices.

FIG. 18A is a diagram showing a fourth example configuration of aprojection system according to the embodiment. The projection system ofthe fourth example is equipped with a plurality of (in the illustratedexample, four) measuring projection devices 100A and 100B and performsposition measurement and video projection on target objects 105 and 106by these measuring projection devices 100A and 100B.

In the fourth example, one measuring projection device (P2) 100Afunctions as a master and the other measuring projection devices (P1,P3, and P4) 100B function as slaves. Although FIG. 18A is drawn in sucha manner that a timing control unit is provided inside the mastermeasuring projection device 100A, the timing control unit may beprovided outside the measuring projection device 100A. The measuringprojection device P2 projects measurement light in a first projectionperiod from the self device as a master or projects measurement light ina first projection period after receiving a timing instruction as aslave. The configuration of the plurality of measuring projectiondevices is not limited to one in which they are connected to each otherby wired communication interfaces to synchronize measurement lightprojection timings, and may be one in which they are connected to eachother by wireless communication interfaces. As a further alternative, aconfiguration is possible in which measurement light projection timingsare synchronized using optical sensors as shown in the third exampleprojection system shown in FIG. 14. Upon completion of the measurementpattern projection by the measuring projection device P2, the measuringprojection devices P1 and P3 located on the two respective sides of itproject measurement patterns in the next projection period. In thismanner, interference between measurement light beams projected by theplurality of measuring projection devices adjacent to each other isprevented.

FIG. 18B is a time chart showing an example of measurement patternprojection times in the fourth example projection system according tothe embodiment. The measuring projection device P2 projects ameasurement pattern in a first projection period (first phase T1). At atiming when the measurement pattern projection by the measuringprojection device P2 has completed, each of the two adjacent measuringprojection devices P1 and P3 projects a measurement pattern in the nextprojection period (second phase T2). At a timing when the measurementpattern projection by the measuring projection devices P1 and P3 havecompleted, the measuring projection device P2 projects a measurementpattern again as the first phase T1. At this time, the measuringprojection device P4 which is located adjacent to the measuringprojection device P3 projects, as the first phase T1, a measurementpattern at the same time as the measuring projection device P2 does.

As described above, since the projection range of the measuringprojection device P2 overlaps with that of each of the measuringprojection devices P1 and P3, the measuring projection device P2projects measurement light at the projection timing (first phase T1)that is different from the projection timing at which each of themeasuring projection devices P1 and P3 projects measurement light. Themeasuring projection devices P1 and P3 project measurement light beamsat the same timing (second phase T2) because the projection range of themeasuring projection device P1 overlaps with that of the measuringprojection device P2 but does not overlap with that of the measuringprojection device P3.

The projection adjustment device 200 can determine order, phases,projection timings, etc. of projection operations of the above-describedplurality of respective measuring projection devices P1 to P4 thatrelate to projection times of the measuring projection devices P1 to P4on the basis of the arrangement of the projection ranges and otherfactors and set them in the measuring projection devices P1 to P4. Aspecific example process that is executed according to the projectionadjustment program in the projection adjustment device 200 will bedescribed later.

Since the four measuring projection devices project measurement lightbeams by turns in the above-described manner, interference between themeasurement light beams can be prevented even though the projectionranges of adjacent measuring projection devices overlap with each other.Furthermore, where two of the projection ranges overlap with each other,position measurement and video projection can be performed in theplurality of measuring projection devices in a shorter time byprojecting measurement light beams by turns using two phases.

FIG. 19A is a diagram showing an example set of projection ranges in thefourth example projection system according to the embodiment. In thefourth example projection system, it is assumed that the projectionranges of the respective measuring projection devices P1 to P4 arearranged as shown in FIG. 19A, for example. In this case, the projectionadjustment device 200 acquires the projection ranges using positionmeasurement information of sets of projection coordinates obtained byposition measurement by each of the measuring projection devices P1 toP4. The projection adjustment device 200 displays a projection range PE1of the measuring projection device P1, a projection range PE2 of themeasuring projection device P2, a projection range PE3 of the measuringprojection device P3, a projection range PE4 of the measuring projectiondevice P4 on the display unit 240 in the form of rectangular figuresshown in the illustrated example. A user can recognize the projectionranges of the plurality of respective measuring projection devices P1 toP4 by seeing a display of such projection position information.

FIG. 19B is a time chart showing a measurement pattern projectionoperation in the fourth example projection system according to theembodiment. In displaying measurement patterns, the measuring projectiondevices P1 to P4 can emit measurement patterns including ID codesindicating device IDs that are unique to the individual devices,respectively. In the illustrated example, ID1, ID2, ID3, and ID4 are setfor the respective measuring projection devices P1, P2, P3, and P4 andthe measuring projection devices P1 to P4 project measurement patternsincluding their own ID codes at their heads, respectively. When themeasuring projection device P2 has projected measurement light, themeasuring projection devices P1 and P3 whose projection ranges overlapwith the projection range of the measuring projection device P2 candetect the measurement light projection by the measuring projectiondevice P2. Thus, at the end of the measurement light projection by themeasuring projection device P2 the measuring projection devices P1 andP3 can recognize that they can project measurement light. As a result,even in a case that the projection ranges of a plurality of measuringprojection devices overlap with each other in a complicated manner, eachmeasuring projection device can recognize measurement light projectionof each of other measuring projection devices and hence can properlyjudge a timing when the self device is to project measurement light.

(Example Projection Adjustment Method)

FIG. 20 is a flowchart showing the procedure of a projection adjustmentmethod of the projection adjustment device according to the embodiment.Here an example procedure for displaying a connection relationshipbetween a plurality of projection ranges and setting projection timingsof the respective measuring projection devices will be described as theprocedure of a process relating to the positional relationship display211 and the projection time setting 212 which are performed according tothe projection adjustment program 221.

In the projection adjustment device 200, the processing unit 210executes the process according to the projection adjustment program 221.First, the projection adjustment device 200 displays a GUI manipulationpicture, that is, a manipulation picture for user manipulation, on thedisplay unit 240 as a user interface (S11). Then the projectionadjustment device 200 executes the following process according to a usermanipulation that has been made. If a user makes a manipulation as aninstruction to perform a measurement, the projection adjustment device200 initializes a count i (counter value) of measuring projectiondevices to “1” (S12) and then causes the ith (in the initial state,first) measuring projection device to perform projector projection(S13). The ith measuring projection device is caused to performabove-described measurement light projection as the projectorprojection. At this time, the projection adjustment device 200 causesthe cameras of the image capture devices of all the measuring projectiondevices to shoot a projection image of the measurement light emittedfrom the ith measuring projection device (S14).

The projection adjustment device 200 judges a connection relationship ofthe projection range of the ith measuring projection device (i.e.,presence/absence of an overlap, a measuring projection device(s) withwhich an overlap is found, etc.) on the basis of image capture resultsof all the measuring projection devices (S15). If a projection rangeoverlap exists, another (or other) measuring projection device succeedsin capturing the measurement light image. A projection range connectionrelationship between the plurality of measuring projection devices canbe judged on the basis of a position(s) of the measuring projectiondevice(s) that has succeeded in capturing the measurement light image.

Then the projection adjustment device 200 sets the count i of measuringprojection devices to i+1 (S16) and judges whether the count i issmaller than the number of measuring projection devices (S17). If thecount i is smaller than the number of measuring projection devices (S17:yes), that is, if there remains a measuring projection device(s) forwhich a projection range connection relationship has not been judgedyet, steps S13 to S17 are executed again in the same manners asdescribed above. That is, the projection adjustment device 200 causesevery measuring projection device to perform, in order, an operation ofprojecting measurement light and having the other measuring projectiondevices shoot its image and judges a projection range connectionrelationship of the measuring projection device concerned. The measuringsteps that each measuring projection device projects measurement lightand all the measuring projection devices perform capture of image may beexecuted either only upon reception of a user manipulation instructionto perform a measurement or every prescribed time automatically.

If the count i becomes equal to the number of measuring projectiondevices (S17: no), the projection adjustment device 200 generatesprojection position information indicating a connection relationshipbetween the projection ranges of the respective measuring projectiondevices. In this example process, as an example, the projectionadjustment device 200 generates a graphical display picture indicating aconnection relationship between the projection ranges (S18). Then theprojection adjustment device 200 draws the graph in the manipulationpicture displayed on the display unit 240 and thereby displays,graphically, the connection relationship between the projection rangesof the respective measuring projection devices (S19).

If the user makes a manipulation instruction to assign projectiontimings automatically, the projection adjustment device 200 calculatesdistances between the projection ranges of the respective measuringprojection devices on the basis of projection position information suchas the graph generated above and sets a start point measuring projectiondevice (S21). A master measuring projection device may be set as a startpoint measuring projection device. Subsequently, the projectionadjustment device 200 traces connected measuring projection devices inorder starting from the start point measuring projection device andassigns, recursively, the measuring projection devices slot numbersindicating order of projection timings (each slot number is assigned tomeasuring projection devices whose projection ranges do not overlap witheach other) (S22). The slot numbers correspond to above-mentioned phasesof projection periods. If the above-described measurement steps S13 toS17 had not completed yet when the user made an automatic assignmentmanipulation instruction, the steps of assigning projection timings tothe respective measuring projection devices may be executed afterexecution of the measurement steps.

Since as described above a connection relationship between theprojection ranges of the respective measuring projection devices isjudged and projection position information indicating the connectionrelationship is generated and displayed, the user can easily recognize apositional relationship between the projection ranges of the pluralityof measuring projection devices. Seeing an image display in which theindividual projection ranges are indicated by figures or the like, agraphical display in which a connection relationship such as overlapsbetween projection ranges is indicated by, for example, nodes andconnection lines, the user can recognize a positional relationshipbetween the plurality of projection ranges at a glance: display ofprojection position information that is high in visibility can beprovided. This makes it possible to effectively assist projectionoperation adjusting work in which the user adjusts, for example, theprojection ranges and/or projection timings of the respective measuringprojection devices.

Furthermore, since a start point measuring projection device isdetermined and projection timings are assigned in order from the startpoint measuring projection device, projection timings such as properprojection order and phases can be set according to an arrangement ofprojection ranges while interference between measurement light beams ofthe plurality of measuring projection devices is prevented. An optimummeasuring projection device that enables an efficient adjustment can beset as a measuring projection device to serve as a start point ofmeasurement light projection.

Various example modes of display of projection position information bythe projection adjustment device 200 will be described below.

FIG. 21A is a diagram showing a first example image display ofprojection ranges of a plurality of measuring projection devices in aprojection adjustment device according to the embodiment. FIG. 21B is adiagram showing a first example graphical display of the projectionranges of the plurality of measuring projection devices in theprojection adjustment device according to the embodiment. The firstexample displays of projection position information are example displaysof projection position information by the projection adjustment device200.

To display projection position information in a manipulation picture ofthe display unit 240, the projection adjustment device 200 performsimage display of projection ranges of a plurality of measuringprojection devices shown in FIG. 21A or graphical display of theprojection ranges of the plurality of measuring projection devices shownin FIG. 21B. In the image display shown in FIG. 21A, in the case wherenine measuring projection devices P1 to P9 are provided as a pluralityof measuring projection devices 100, projection ranges PE1 to PE9 of therespective measuring projection devices P1 to P9 are representedschematically by rectangular figures, respectively. Seeing this imagedisplay, the user can visually recognize an arrangement of and aconnection relationship between the projection ranges PE1 to PE9 easily.Whereas projection ranges 351 of the respective measuring projectiondevices are displayed, double projection ranges 352 each of which is anoverlap of two projection ranges, triple projection ranges 353 each ofwhich is an overlap of three projection ranges, quadruple projectionranges 354 each of which is an overlap of four projection ranges, etc.may be displayed in an emphasized manner by discriminating them usingdifferent colors, patterns, or the like. In this manner, the user isallowed to easily recognize overlaps between projection ranges andmanners of overlap by seeing an image display.

The graphical display shown in FIG. 21B is a graph in which positions ofthe projection ranges of the respective measuring projection devices P1to P9 are displayed in the form of nodes 361 such as circular marks andnodes 361 representing projection ranges that overlap with (i.e., areconnected to) each other are connected to each other by a connectionline 362. Seeing this graphical display, the user can recognize, at aglance, a connection relationship between the projection ranges of therespective measuring projection devices. Phases T1 to T4 indicatingmeasurement light projection timings of the respective measuringprojection devices P1 to P9 may also be displayed. Where a mastermeasuring projection device is set, it may be displayed so as to bediscriminated by a color, pattern, mark, or the like. Although thegraphical display of FIG. 21B is of a non-directional graph in which theconnection lines 362 have no direction, a directional graph may beemployed in which the connection lines 362 are given arrows or the likeindicating interference directions in the case where interferencebetween measurement light beams is only in one direction. A directionalgraph may be employed in, for example, a case that target objects havecomplicated shapes and a first measuring projection device interfereswith a second measuring projection device but the second measuringprojection device does not interfere with the first measuring projectiondevice (i.e., in the opposite direction).

Specific examples of how to set a start point measuring projectiondevice among a plurality of measuring projection devices and how toassign projection timings to the respective measuring projection devicesautomatically will be described using the example displays of FIGS. 21Aand 21B. The projection adjustment device 200 generates the graphicdisplay of FIG. 21B from the image display of FIG. 21A. Among the nodes361 of the respective measuring projection devices, two adjacent nodes(measuring projection devices) whose projection ranges overlap with eachother are connected to each other by a connection line 362. Theprojection adjustment device 200 measures distances between each node(measuring projection device) and the other nodes, extracts a node whosedistance from a farthest node is shortest, that is, a node whose numberof steps taken to reach every other nodes is smallest, and makes thisnode a start point measuring projection device. As for distances betweena plurality of nodes, a shortest route can be calculated by Dijkstra'salgorithm or the like. A start point measuring projection device may beset as a master measuring projection device. In the illustrated example,the central measuring projection device P5 is set as a start pointmeasuring projection device because it can reach every other node(measuring projection device) by one step. Then the projectionadjustment device 200 assigns, in order, recursively, the nodes slotnumbers indicating order of projection timings starting from the startpoint measuring projection device P5 (each slot number is assigned tonodes whose projection ranges do not overlap with each other). In theillustrate example, a first phase T1 is assigned to the measuringprojection device P5, a second phase T2 is assigned to the measuringprojection devices P4 and P6, a third phase T3 is assigned to themeasuring projection devices P2 and P8, and a fourth phase T4 isassigned to the measuring projection devices P1, P3, P7, and P9.

The projection adjustment device 200 performs the above-describedprocessing of measuring projection ranges of the plurality of respectivemeasuring projection devices in real time and updates the image displayor the graphic display of projection position information according to acurrent arrangement of the measuring projection devices. By performingthis processing, the projection adjustment device 200 assists projectionoperation adjusting work in which the user adjusts, for example, theprojection ranges of the respective measuring projection devices and/orthe projection timing manually.

FIG. 22A is a diagram showing a second example image display ofprojection ranges of a plurality of measuring projection devices in theprojection adjustment device according to the embodiment. FIG. 22B is adiagram showing a second example graphical display of the projectionranges of the plurality of measuring projection devices in theprojection adjustment device according to the embodiment. The secondexample displays of projection position information are example displaysof a case that the projection ranges of part of the measuring projectiondevices are changed from the state of the first example shown in FIGS.21A and 21B.

As shown in the image display of FIG. 22A, assume that the projectionrange PE9 of the measuring projection device P9 which is located at thebottom-right position in the figure is displaced downward and, as aresult, comes not to overlap with the projection range PE6 of themeasuring projection device P6. In this case, in the graphic displayshown in FIG. 22B, no connection line is displayed between the node ofthe measuring projection device P9 and the node of the measuringprojection device P6, whereby the user can recognize that nointerference occurs and no connection relationship exists between themeasuring projection devices P6 and P9. Seeing the image display shownin FIG. 22A, the user can recognize that a non-continuous region existsbetween the projection ranges PE6 and PE9. While seeing the display ofthe projection position information displayed by the projectionadjustment device 200, the user can recognize the arrangement of theprojection ranges of the respective measuring projection devices andadjust the projection ranges so as to obtain a proper arrangement. It isalso possible for the user to set or adjust order, phases, projectiontimings, etc. of projection operations of the plurality of measuringprojection devices on the basis of a display of projection positioninformation.

FIG. 23A is a diagram showing a third example image display ofprojection ranges of a plurality of measuring projection devices in theprojection adjustment device according to the embodiment. FIG. 23B is adiagram showing a third example graphical display of the projectionranges of the plurality of measuring projection devices in theprojection adjustment device according to the embodiment. The thirdexample displays of projection position information are example displaysof projection position information displayed by the projectionadjustment device 200.

To display projection position information in a manipulation picture ofthe display unit 240, the projection adjustment device 200 performsimage display of projection ranges of a plurality of measuringprojection devices shown in FIG. 23A or graphical display of theprojection ranges of the plurality of measuring projection devices shownin FIG. 23B. In the image display shown in FIG. 23A, in the case wherefour measuring projection devices P1 to P4 are provided as a pluralityof measuring projection devices 100, projection ranges PE1 to PE4 arerepresented schematically by rectangular figures, respectively. In thisexample, a double projection range 371 (region A) which is an overlap oftwo projection ranges and a triple projection range 372 (region B) whichis an overlap of three projection ranges are displayed in an emphasizedmanner by discriminating them using different colors, patterns, or thelike.

In the graphical display shown in FIG. 23B, positions of the projectionranges of the respective measuring projection devices P1 to P4 aredisplayed in the form of nodes 361 such as circular marks and nodes 361representing projection ranges that overlap with (i.e., are connectedto) each other are connected to each other by a connection line 362 andoverlaps are displayed in an emphasized manner in the form of aplurality of kinds of nodes so that the overlapping states of projectionranges can be recognized in a more understandable manner. In theillustrated example, a region A is represented by an overlap node 381(double projection range) and a region B is represented by an overlapnode 382 (triple projection range). The degrees of overlapping, thesizes of overlap regions, etc. of the overlap nodes 381 and 382representing the respective overlaps are displayed so as to bediscriminated from each other by colors, patterns, shapes, or the like.

Seeing an image display or a graphic display as described above, theuser can easily recognize various kinds of states of projection rangessuch as positions where projection ranges of respective measuringprojection devices overlap with each other, the number of overlapsbetween projection ranges, and a positional relationship betweenmeasuring projection devices whose projection ranges overlap with eachother. In particular, since overlaps are represented by nodes of agraphic display, the user can recognize the degrees of overlapping,overlapping relationships, etc. of projection ranges at a glance.

FIG. 24 is a diagram showing a fourth example image display ofprojection ranges of a plurality of measuring projection devices in theprojection adjustment device according to the embodiment. FIG. 25 is adiagram showing a fifth example image display of projection ranges of aplurality of measuring projection devices in the projection adjustmentdevice according to the embodiment. The fourth example image display andthe fifth example image display of projection position information areexample displays of projection position information for assistingprojection operation adjusting work of the user.

To display projection position information in a manipulation picture ofthe display unit 240, the projection adjustment device 200 performsimage display of projection ranges of a plurality of measuringprojection devices shown in FIG. 24. In the illustrated example, theprojection ranges of a plurality of measuring projection devices arerepresented schematically by rectangular figures, respectively, andphases T1 to T4 of projection timings of the respective measuringprojection devices are shown.

Where rectangular projection ranges, having four sides in the outercircumference, of a plurality of measuring projection devices areadjacent to each other, the maximum number of overlaps is four and henceall the measuring projection devices can perform projection if fourphases T1 to T4 are set as projection timings. Quadruple projectionranges 391 may be displayed so as to be discriminated from each otherusing different colors, patterns, or the like so that the user canrecognize the overlap regions between projection ranges and the numberof overlaps visually.

Assume that the fifth example shown in FIG. 25 has been obtained by theuser's adjusting and displacing the projection ranges of part of themeasuring projection devices from the state of the forth example shownin FIG. 24. In this case, the maximum number of overlaps betweenprojection ranges can be changed to three by shifting the projectionranges of six measuring projection devices (i.e., three second-stage(from the top) projection ranges and three fourth-stage projectionranges) in the left-right direction in the figure. In this case,projection by all the measuring projection devices is enabled by settingthree phases T1 to T3 as projection timings. Triple projection ranges392 may be displayed so as to be discriminated using different colors,patterns, or the like so that the user can recognize the overlap regionsbetween projection ranges and the number of overlaps visually. Positionmeasurement and video display can be performed efficiently by adjustingthe projection ranges of the plurality of measuring projection devicesin the above-described manner so that they can cover the entire areawith the three phases.

The above-described image display can assist projection operationadjusting work so that the user can set proper projection ranges andprojection timings. Incidentally, where a display of projection positioninformation has quadruple projection ranges that necessitate 4-phaseprojection, it is possible to assist the user in setting an arrangementof projection ranges easily that enables a 3-phase projection operationby making a warning display for the quadruple projection ranges usingconspicuous colors, marks, or the like. It is also possible to assistthe user by calculating movement directions, movement distances, etc. inadjustment of positions of projection ranges.

As described above, the projection adjustment program according to theembodiment is a projection adjustment program that allows a computer toexecute a process relating to adjustment of projection operations of aplurality of measuring projection devices 100 in a projection systemincluding the plurality of measuring projection devices 100 asprojection devices for performing position measurement and projection ontarget objects 105 and 106. For example, the projection adjustmentprogram is run by the projection adjustment device 200 which isconnected to the measuring projection devices 100 of the projectionsystem. The program allows the process to cause a first projectiondevice of the projection system to project invisible measurement lightonto the target objects; cause a second projection device of theprojection system to receive reflection light, reflected from the targetobjects, of the measurement light; and judges a connection relationshipof a projection range of the first projection device on the basis of thereceived reflection light of the measurement light. Furthermore, theprogram allows the process to execute the connection relationshipjudging step on all processing target projection devices, and generatesprojection position information indicating a connection relationshipbetween the projection ranges of the respective projection devices ofthe projection system and displays the generated projection positioninformation on a display unit.

With this program, in arranging a plurality of projection devices, auser can easily recognize a positional relationship between a pluralityof projection ranges such as presence/absence of overlap betweenprojection ranges, overlapping relationships of the projection ranges,and the number of overlaps. Furthermore, it becomes possible to assist,effectively, projection operation adjusting work so that the user canset proper projection ranges and projection timings.

The projection adjustment program allows the process to generate animage display representing the projection ranges of the measuringprojection devices as a display of the projection position information.With this measure, the user can easily recognize a positionalrelationship between a plurality of projection ranges such as anarrangement of the projection ranges of the plurality of measuringprojection devices and overlapping relationships between projectionranges by an image display such as a display of figures representing therespective projection ranges.

The projection adjustment program allows the process to generate agraphical display including nodes indicating positions of the projectionranges of the measuring projection devices, respectively, and connectionlines indicating a connection relationship between the nodes thatreflects overlaps between the projection ranges, as a display of theprojection position information. With this measure, the nodes of thegraphical display allow the user to recognize an arrangement of theprojection ranges of the plurality of respective measuring projectiondevices and the connection lines allow the user to easily recognize aconnection relationship between the projection ranges (i.e., overlappingstates).

If the projection ranges of a plurality of measuring projection deviceshave an overlap, the projection adjustment program allows the process todisplay the overlap in an emphasized manner. With this measure, the usercan recognize an overlap between projection ranges easily at a glanceand hence can recognize a connection relationship between the projectionranges easily.

If the projection ranges of a plurality of projection devices have anoverlap, the projection adjustment program allows the process to displayan overlap node indicating an overlapping state of the projection rangesat the overlap. With this measure, the user can recognize overlapsbetween projection ranges and overlapping states such as the degrees ofoverlapping easily at a glance and hence can recognize a connectionrelationship between the projection ranges easily.

The projection adjustment program further allows the process tocalculate distances between the projection ranges of the respectiveprojection devices on the basis of the projection position informationindicating the connection relationship between the projection ranges;and to set, as a start point projection device, a projection devicehaving a shortest distance to the other projection devices. With thismeasure, an optimum measuring projection device that provides highestefficiency can be set on the basis of an arrangement of the projectionranges of the respective measuring projection devices as a measuringprojection device to serve as a start point of measurement lightprojection such as a master measuring projection device.

The projection adjustment program allows the process to assignprojection timings to the measuring projection devices in orderrecursively starting from the start point projection device, eachprojection timing being assigned to measuring projection devices whoseprojection ranges do not overlap with each other. With this measure,proper projection timings such as projection order and phases that aresuitable for an arrangement of the projection ranges can be set whileinterference between measurement light beams of the plurality ofmeasuring projection devices is prevented.

The projection adjustment method according to the embodiment is aprojection adjustment method of the projection adjustment device 200which executes a process relating to adjustment of projection operationsof a plurality of projection devices 100 as projection devices thatperform position measurement and projection on target objects 105 and106 in a projection system including the plurality of projection devices100. The projection adjustment method causes a first projection deviceof the projection system to project invisible measurement light onto thetarget objects; causes a second projection device of the projectionsystem to receive reflection light, reflected from the target object, ofthe measurement light; and judges a connection relationship of aprojection range of the first projection device on the basis of thereceived reflection light of the measurement light. Furthermore, theprojection adjustment method executes the connection relationshipjudging step on all processing target projection devices; and generatesprojection position information indicating a connection relationshipbetween the projection ranges of the respective projection devices ofthe projection system and displays the generated projection positioninformation on a display unit. This projection adjustment method allowsthe user to easily recognize a positional relationship between aplurality of projection ranges in arranging a plurality of projectiondevices.

Although the various embodiments have been described above withreference to the drawings, it goes without saying that the invention isnot limited to those examples. It is apparent that those skilled in theart could conceive various changes or modifications within the confinesof the claims, and they are naturally construed as being included in thetechnical scope of the invention. Constituent elements of theabove-described embodiments may be combined in a desired manner withoutdeparting from the spirit and scope of the invention.

The present application is based on Japanese Patent Application No.2018-095728 filed on May 17, 2018, the disclosure of which is invokedherein by reference.

INDUSTRIAL APPLICABILITY

The present disclosure is useful in providing a projection adjustmentprogram and a projection adjustment method which allow, in arranging aplurality of projection devices, a user to easily recognize a positionalrelationship between a plurality of projection ranges.

DESCRIPTION OF REFERENCE SIGNS

100, 100A, 100B, 140, 160A, 160B: Measuring projection device

101: Image capture device

102: Infrared light projection device

103: Computing device

104: Visible light projection device

105: First target object

106: Second target object

111: Lens optical system

112: Infrared LED light source

113: Display device

131: Setting input unit

142: Projection device

150: Timing control unit

151: Timing generator

152: External synchronization interface (I/F)

153: Optical sensor

200: Projection adjustment device

210: Processing unit

220: Storage unit

221: Projection adjustment program

230: Communication interface (I/F)

240: Display unit

250: Monitor

260: Input unit

401: Image input unit

402: Pattern decoding unit

405: Coordinates conversion unit

407: Coordinates interpolation unit

408: Content generation unit

410: Image output unit

411: Pattern generation unit

The invention claimed is:
 1. A non-transitory computer-readable storagemedium that stores a projection adjustment program, the projectionadjustment program, when executed by a processor, causing a computer toexecute a process relating to adjustment of projection operations of aplurality of projection devices, the plurality of projection devicesconfigured to perform position measurement and projection on a targetobject in a projection system, the projection system comprising theplurality of projection devices, the process comprising: causing a firstprojection device of the projection system to project invisiblemeasurement light onto the target object; causing a second projectiondevice of the projection system to receive reflection light of themeasurement light, the reflection light being reflected from the targetobject; judging a connection relationship of a projection range of thefirst projection device based on the received reflection light of themeasurement light; executing a process of the judging of the connectionrelationship on all processing target projection devices; and generatingprojection position information indicating a connection relationshipbetween projection ranges of the respective projection devices of theprojection system and displaying the projection position information ona display, wherein, in the generating, a graphical display is generatedas a display of the projection position information, the graphicaldisplay comprising nodes indicating positions of the projection devices,respectively, and connection lines indicating a connection relationshipbetween the nodes that reflects overlaps between the projection rangesof the projection devices.
 2. The non-transitory computer-readablestorage medium according to claim 1, wherein an image displayrepresenting the projection ranges of the projection devices isgenerated as the display of the projection position information.
 3. Thenon-transitory computer-readable storage medium according to claim 1,wherein, when the projection ranges of the plurality of projectiondevices have an overlap, the overlap is displayed in an emphasizedmanner.
 4. The non-transitory computer-readable storage medium accordingto claim 1, wherein, when the projection ranges of the plurality ofprojection devices have an overlap, an overlap node indicating anoverlapping state of the projection ranges is displayed at the overlap.5. The non-transitory computer-readable storage medium according toclaim 1, wherein the process further comprises: calculating distancesbetween the projection ranges of the respective projection devices basedon the projection position information indicating the connectionrelationship between the projection ranges; and setting, as a startpoint projection device, a projection device having a shortest distanceto other projection devices.
 6. The non-transitory computer-readablestorage medium according to claim 5, wherein projection timings areassigned to measuring projection devices in order recursively startingfrom the start point projection device, with the projection timingsbeing assigned to the measuring projection devices whose projectionranges do not overlap with each other.
 7. A projection adjustment methodof a projection adjustment device, the projection adjustment deviceconfigured to execute a process relating to adjustment of projectionoperations of a plurality of projection devices, the plurality ofprojection devices configured to perform position measurement andprojection on a target object in a projection system, the projectionsystem comprising the plurality of projection devices, the projectionadjustment method comprising: causing a first projection device of theprojection system to project invisible measurement light onto the targetobject; causing a second projection device of the projection system toreceive reflection light, reflected from the target object, of themeasurement light; judging a connection relationship of a projectionrange of the first projection device based on the received reflectionlight of the measurement light; executing a process of the judging ofthe connection relationship on all processing target projection devices;and generating projection position information indicating a connectionrelationship between the projection ranges of the respective projectiondevices of the projection system and displaying the generated projectionposition information on a display, wherein, in the generating, agraphical display is generated as a display of the projection positioninformation, the graphical display comprising nodes indicating positionsof the projection devices, respectively, and connection lines indicatinga connection relationship between the nodes that reflects overlapsbetween the projection ranges of the projection devices.
 8. A projectionsystem, comprising: a plurality of projection devices configured toperform position measurement and projection on a target object; and aprojection adjustment device including a memory, the memory storing aprojection adjustment program, the projection adjustment program, whenexecuted by the projection adjustment device, causing the projectionadjustment device to execute a process, the process relating toadjustment of projection operations of the plurality of projectiondevices, the process including: causing a first projection device of theprojection system to project invisible measurement light onto the targetobject; causing a second projection device of the projection system toreceive reflection light of the measurement light, the reflection lightbeing reflected from the target object; judging a connectionrelationship of a projection range of the first projection device basedon the received reflection light of the measurement light; executing aprocess of the judging of the connection relationship on all processingtarget projection devices; and generating projection positioninformation indicating a connection relationship between projectionranges of the respective projection devices of the projection system anddisplaying the projection position information on a display, wherein, inthe generating, a graphical display is generated as a display of theprojection position information, the graphical display comprising nodesindicating positions of the projection devices, respectively, andconnection lines indicating a connection relationship between the nodesthat reflects overlaps between the projection ranges of the projectiondevices.